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We develop a direct Lyapunov method for the almost sure open-loop 
stabilizability and asymptotic stabilizability of controlled degenerate dif- 
fusion processes. The infinitesimal decrease condition for a Lyapunov 
function is a new form of Hamilton- Jacobi-Bellman partial differential in- 
equality of 2nd order. We give local and global versions of the First and 
Second Lyapunov Theorems assuming the existence of a lower semicontin- 
uous Lyapunov function satisfying such inequality in the viscosity sense. 
An explicit formula for a stabilizing feedback is provided for affine systems 
with smooth Lyapunov function. Several examples illustrate the theory. 
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1 Introduction 

For controlled diffusion processes in M N 



there are various possible notions of Lyapunov stability of an equilibrium, say 
the origin. The stability in probability has been studied for a long time, we recall 
here the contributions of Kushner |3ni|2], Has'minskii [35], and the recent book 
of Mao 33] for uncontrolled systems, and the work of Florchinger |2()l 1211 l2~2"] 
and Deng, Krstic, and Williams ^Ij on feedback stabilization for (CSDE), see 
also the references therein. The almost sure exponential stability was introduced 
and studied by Kozin |2B]> see also [25], and it implies that for each fixed sample 
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project "Partial differential equations and control theory". 



Abstract 




dX t = f(X t ,at)dt + a(X t ,a t )dB t , a t € A, t > 0, 



in a set of probability 1 the (uncontrolled) system is exponentially stable in the 
usual sense. In this paper we consider a property that we call almost sure 
stability , or uniform stability with probability 1. For an uncontrolled system 
it says that for any 77 > there exists 5 > such that, for any x with |x| < S, 
the process satisfies \X t \ < n for all i > almost surely. Equivalently, for some 
increasing, continuous function 7 null at 0, and for small 

\X t \ < 7(1x1) Vi>0 a.s. (1) 

This property describes a behaviour very similar to a stable deterministic sys- 
tem, it is stronger than stability in probability and pathwise stability, and in 
fact it is never verified by a nondegenerate process. More precisely, we study 
the almost sure open loop stabilizability of (CSDE), namely, that for each x as 
above there exists an admissible control function whose trajectory X verifies 
a.s. \X t \ < r\ (and \X t \ < 7(|a;|)) for all t. If, in addition, lim t ^ +00 X t — a.s., 
we say the system is a.s. (open loop) asymptotically stabilizable. For determin- 
istic systems (c = 0) the last property reduces to the well-known asymptotic 
controllability. 

We follow the Lyapunov Direct Method and find that the infinitesimal de- 
crease condition to be satisfied by a Lyapunov function V for our problem is 



max {-DV(x)- f(x,a)- trace \a(x,a)D 2 V(x)]} >l(x), (2) 

a£A, a(x,a) T DV(x)=0 

with I > for mere Lyapunov stability and I > for x ^ for asymptotic 
stability, where a :— aa T /2. This is not a standard Hamilton- Jacobi-Bellman 
inequality, because the constraint on the control a depends on V. In fact it 
should be rather viewed as a system of PDEs and inequalities, that in the 
special case of uncontrolled diffusion, i.e., a = o~(x), reads 

max a6i i {—DV(x) ■ f(x, a)} — trace [a(x)D 2 V(x)] > l{x) , , 

<Ti(x) • DV{x) = Vt, " [ ' 

where cr, denotes the i-th column of the matrix a. Note that the first condition of 
the system is the Hamilton- Jacobi-Bellman inequality that defines the Lyapunov 
functions for the stability in probability. The other conditions mean that there 
is diffusion only in the directions tangential to the level sets of V. To motivate 
the infinitesimal decrease condition J2J we take V of class C 2 and suppose that 
for a fixed control function dV(X t )/dt < l{X t ). Then Ito's formula gives 

[DV(X t ) ■ f(X t , a t ) + trace (a(X t )D 2 V(X t ))] dt + a T (X t )DV(X t )dB t 

< l(X t ). 

Now the properties of the Brownian motion lead to the conditions 

DV(X t )- f(X u a t ) + trace (a(X t )D 2 V(X t )) < l(X t ), 

a T (X t )DV(X t ) = 0, 

and the existence of a control at verifying this is clearly related to (JSJl • 

We define a Lyapunov function for the a.s. stability as a lower semicontinu- 
ous proper function V, continuous at and satisfying J3J) in the viscosity sense, 
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and we call it strict Lyapunov function if I > off 0, see the Definitions [3] and 0] 
below. Our main results are the natural extensions to the controlled diffusions 
of the First and Second Lyapunov Theorems: 

the existence of a local Lyapunov function implies the a.s. (open loop) stabi- 
lizability of (CSDE); a strict Lyapunov function implies the a.s. (open loop) 
asymptotic stabilizability. 

The same proof provides their global versions as well: if V satisfies © in M N \ 
{0} then (CSDE) is also a.s. (open loop) Lagrange stabilizable, i.e., for all 
initial points x there is a control such that (JJJ holds, and if V is strict then 
the system is globally a.s. (open loop) asymptotically stabilizable. We also give 
sufficient conditions for the stability of viable (controlled invariant) sets more 
general than an equilibrium point, and for the a.s. exponential stability. 

These facts are much easier to prove if the Lyapunov function is smooth, 
but this assumption is not necessary and would limit considerably their appli- 
cability. The nonexistence of smooth Lyapunov functions is well known in the 
deterministic case, see j2Hl E] for stable uncontrolled systems and the surveys 
[411 |S] for asymptotically stable controlled systems. Here we give an example 
of an uncontrolled degenerate diffusion process that is a.s. stable but cannot 
have a continuous Lyapunov function (Example \I\m Section 6). Moreover, in a 
companion paper |12j the second author proves a converse Lyapunov theorem, 
stating that any a.s. stabilizable system (CSDE) has a l.s.c. local viscosity 
Lyapunov function. 

All the results listed above are about open loop a.s. stabilizability. They 
raise the question of the existence of a stabilizing feedback. Here we give an 
answer only for affine systems with a smooth strict Lyapunov function. We 
adapt Sontag's method [213 to the stochastic setting and find an explicit formula 
for a feedback that renders the system almost surely asymptotically stable. The 
feedback stabilizability of controlled diffusions in case of nonsmooth Lyapunov 
functions seems considerably harder and we are not aware of any paper on the 
subject. 

In the last section we study some simple applications and examples. For 
instance, we consider a deterministic, asymptotically controllable system X t — 
f(X t ,at) with Lyapunov pair (V,L) and look for conditions on a stochastic 
perturbation that keep the system a.s. stabilizable with the same Lyapunov 
function V for some I < L. 

Our proof of the first Lyapunov-type theorem is based on the observation 
that the infinitesimal decrease condition (0 has the rescaling property of the 
geometric PDEs arising in the level set approach to front propagation (see, e.g., 
[51 138) and the references therein), and on a recent result of the first author and 
Jensen on the viability, or controlled invariance, of general closed sets for 
controlled diffusions (see 0] and the references therein for earlier work on 
viability for stochastic processes). For the second Lyapunov-type theorem we 
use also martingale inequalities and other properties of diffusions. 

The first Lyapunov-type theorem on local a.s. stabilizability was announced 
in |S] where we presented the simpler proof for uncontrolled processes. In the 
forthcoming paper |I3| the second author shows that the existence of a contin- 
uous viscosity solution of the Hamilton- Jacobi-Bcllman inequality 

max {— DV(x) ■ f(x, a) — trace [a(x, a)D 2 V(x)\ } > l(x), 
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implies the open loop stabilizability in probability of (CSDE). Converse theo- 
rems in this setting are under investigation. 

We conclude with some additional references. Nonsmooth Lyapunov func- 
tions for uncontrolled diffusion processes were studied by Ladde and Laksh- 
mikantham |32| with Dini-type derivatives along sample paths, and by Aubin 
and Da Prato by means of a stochastic contingent epiderivative. Recently 
Arnold and Schmalfuss pQ gave an extension of Lyapunov's Second Method to 
random dynamical systems. Turning to deterministic controlled systems, we 
recall that Soravia |43| gave direct and inverse Lyapunov theorems for the open 
loop stabilizability by means of viscosity solutions (in the more general context 
of differential games), Sontag and Sussmann [23021 did it for the asymptotic 
controllability (i.e., asymptotic open loop stabilizability) by using Dini direc- 
tional derivatives. Viscosity methods for stability problems were also used in 
[271 1441 l2~3"] . There is a large literature on feedback stabilization, see [2"1 1401 UK] . 
the surveys |41l 1141 Ifi]. and the references therein. We refer to ^3 Ej for the 
basic theory of viscosity solutions, and to [SSI HO 023 QEj for its applications to 
deterministic and stochastic optimal control. 

The paper is organized as follows. In Section 2 we give the main definitions 
and state the first and second Lyapunov-type theorems. Section 3 recalls some 
viability theory and then gives the proofs of the two main theorems. Section 4 
is on feedback stabilization of affine systems with smooth Lyapunov functions. 
Section 5 contains some extensions to exponential stability, general equilibrium 
sets, and target problems. Section 7 is devoted to the examples. 

2 Lyapunov functions for a.s. stabilizability and 
asymptotic stabilizability 

We consider a controlled Ito stochastic differential equation: 



where B t is an M -dimensional Brownian motion. Throughout the paper we 
assume that /, a are continuous functions defined in M N x A, where A is a 
compact metric space, which take values, respectively, in 1R N and in the space 
of N x M matrices, and satisfying 



\f(x,a)-f(y,a)\ + \\a(x,a)-a(y,a)\\<C\x-y\, \fx,yeM N , Va e A. (4) 



We adopt the definition of admissible control function, or admissible system, of 
Haussmann and Lepeltier [23 (Definition 2.2, page 853). For a given x <E M 
we denote with A x the set of admissible control functions, with a. its generic 
element (although it is not a standard function M — ► A), and with X. the 
corresponding solution of (CSDE). We define 




dX t = f(X t ,a t )dt + a(X u a t )dBt, t > 0, 



a(x,a) := - 



a(x, a)a(x, a) T 



and assume 



{(a(x, a), f(x, a)) : a S A} is convex for all x € M 



(5) 
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Definition 1 (almost sure stabilizability). The system (CSDE) is almost 
surely (open- loop Lyapunov) stabilizable at the origin if for every rj > there 
exists 8 > such that, for any initial point x with \x\ < 6, there exists an 
admissible control function a. £ A x whose corresponding trajectory X. verifies 
\Xt \ < V for all t > almost surely. 

The system is almost surely (open-loop) Lagrange stabilizable, or it has the 
property of uniform boundedness of trajectories, if for each R > there is 
S > such that for any initial point x with \x\ < R, there exists an admissible 
control function a. £ A x whose corresponding trajectory X. verifies \X t \ < S 
for all t > almost surely. 

Remark. The a.s. stabilizability implies that the origin is a controlled equilib- 
rium of (CSDE), i.e., 

3a £ A : /(0,a) = 0, cr(0,a) = 0. 

In fact, the definition gives for any e > an admissible control such that the 
corresponding trajectory starting at the origin satisfies a. s. \X t \ < e for all t, so 
Ej + °° \X t \e~ xt dt < f for any A > 0. Then inf Q eAx E x / + °° \X t \ e - xt dt = 0. 
The convexity assumption Q and an existence theorem for optimal controls |2(i| 
imply that the inf is attained, and the minimizing control produces a trajectory 
satisfying a.s. \X t \ = for all t > 0. The conclusion follows from standard 
properties of stochastic differential equations. 

Remark. As it is common in the modern deterministic stability theory, the 
previous definitions can be reformulated in terms of the comparison functions 
introduced by Hahn [2I|. We will use the class IC of continuous functions 7 : 
[0, +00) — > [0, +00) strictly increasing and such that 7(0) = 0, and the class /Coo 
of functions 7 £ IC such that lim r ^ +00 7(7-) = +00. The system (CSDE) is a.s. 
(open-loop) stabilizable at if there exists 7 £ IC and 5 > such that for any 
starting point x with |x| < S 

3 a. eAx: |A F t |<7(|x|) > a.s, (6) 

where X t is the trajectory corresponding to a.. If © holds for some 7 £ /Coo and 
for all x £ M N , then the system is also a.s. (open- loop) Lagrange stabilizable. 

Definition 2 (a.s. asymptotic stabilizability). The system (CSDE) is 
almost surely (open loop) locally asymptotically stabilizable (or a. s. locally 
asymptotically controllable ) at the origin if for every rj > there exists 5 > 
such that, for all \x\ < 5, there exists an admissible control function a. £ A x 
whose corresponding trajectory X. verifies almost surely 

\X t \<T) Vi>0, lim \X t \=0. 

t — >+oo 

The system is a. s. (open loop) globally asymptotically stabilizable (or a. s. 
asymptotically controllable ) at the origin if there is 7 £ /Coo and for all x £ M N 
there exists a. £ A x whose trajectory X. satisfies almost surely 

\X t \ < 7(|2;|) Vt > 0, lim \X t \ = 0. 

t— »+oo 
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Next we give the appropriate definition of Lyapunov function for the study 
of almost sure stabilizability. We recall the definition of second order semijet of 
a l.s.c. function V at a point x: 

J 2 '~V(x) := { (p, Y) 6 M N x S(N) : for y x 

V(y) > V(x) +p-(y-x) + ^(y-x)- Y(y - x) + o(\y - x| 2 )| . 

Definition 3 (control Lyapunov function). Let O C M be an open set 
containing the origin. A function V : O — > [0, +oo) is a control Lyapunov 
function for the a.s. stability of (CSDE) if 

(i) V is lower semicontinuous, 

(ii) V is continuous at and positive definite, i.e., V(0) = and V(x) > 
for all x 7^ 0, 

(Hi) V is proper, i.e., lim\ x \^ +00 V(x) = +oo or, equivalently, the level sets 
{x\V(x) < fi} are bounded for every fj, £ [0, oo); 

(iv) for all x € 0\ {0} and (p, Y) G J 2 '~V{x) there exists a £ A such that 

a(x,a) T p — and — p ■ f(x, a) — trace [a(x, a)Y] > 0. (7) 

Remark. The conditions (ii) and (Hi) in the previous definition can be stated 
as 

3 7 i,72 G /Co, : li(\x\) < V(x) < l2 (\x\) Vx € M N . (8) 

Therefore the level sets {V(x) < /i} of the Lyapunov function form a basis of 
neighborhoods of 0. 

Remark. If the dispersion matrix a does not depend on the control, then 
condition (iv) can be reformulated as follows: 

V is a solution in viscosity sense in O \ {0} of the system: 

f a(x) T DV(x) = 

| maxogA {— DV(x) ■ f(x, a) - trace [a(x, a)D 2 V(x)] } > 

In the general case, we can observe that if the condition (iv) holds, then in 
particular V is a viscosity supersolution of 

max {-DV(x) ■ f(x, a) - trace [a(x, a)D 2 V(x)] } = (9) 

Moreover, if the function V is at least differentiable, then the condition (iv) can 
be stated more concisely as: 

V is a supersolution in viscosity sense in O \ {0} of the equation: 
m&x {aeA | a (x.a) T DV(x)=o} {- DV (x) ■ f(x, a) - trace [a(x, a)D 2 V(x)] } = 

Definition 4 (strict control Lyapunov function). A function V : O — > 
[0, +oo) is a strict control Lyapunov function for the a.s. stability of (CSDE) 
if it satisfies conditions (i), (ii), (Hi) in the Definition^ and 
(iv) 1 for all x € \ {0} and (p,Y) £ 3 2 ~V(x) there exists a £ A such that 

a T (x,a)p — and — p ■ f(x, a) — trace [a(x, a)Y] — l(x) > (10) 

for some positive definite and Lipschitz continuous I : O — > 1R. 
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Remark. In the inequality in (iv)' we could take 

p ■ f(x,a) — trace [a(x, a)Y] — l(x,a) > 

for some continuous !:Ox4-> M, Lipschitz continuous in x uniformly in a, 
with l(x,A) convex for all x £ O, and such that l(x) := min ae A l(x, a ) lii posi- 
tive definite. However, this would not increase the generality of the definition, 
because V would also satisfy condition (|10J) with I replaced by I. 

Our main results are the following versions for stochastic controlled systems of 
the First and the Second Lyapunov Theorem. 

Theorem 5 (a.s. stabilizability). Assume 0), and the existence of a 
control Lyapunov function V . Then 

(i) the system (CSDE) is almost surely stabilizable at the origin; 

(ii) if, in addition, the domain O ofV is all JR N , the system is also a.s. Lagrange 
stabilizable and for all x £ M N there exists a. £ A x such that the corresponding 
trajectory X satisfies 

\X t \ <7r 1 (72(|^D) Vi>0 a.s. (11) 
with 71,72 £ ICcc verifying @). 

Theorem 6 (a.s. asymptotic stabilizability). Assume (0), and the 
existence of a strict control Lyapunov function V. Then 

(i) the system (CSDE) is a.s. locally asymptotically stabilizable at the origin; 

(ii) if, in addition, the domain O of V is all M N , the system is a.s. globally 
asymptotically stabilizable. 

3 A viability theorem and the proofs of stabiliz- 
ability 

In this section we prove the Theorems and El Our main tool is a recent result 
in 1 1 1 1 about the almost sure viability (named also controlled invariance and 
weak invariance) of an arbitrary closed set for a controlled diffusion process. 
(See [30] and the references therein for earlier related results). 

Definition 7 (viable set). A closed set K C LR N is viable or controlled invari- 
ant or weakly invariant for the stochastic system (CSDE) if for all initial points 
x £ K there exists an admissible control a. £ A x such that the corresponding 
trajectory X satisfies X t £ K for all t > almost surely. 

It is easy to see from its very definition that the a.s. stabilizability follows from 
the viability of all the sublevel sets of any function satisfying the conditions 
(i) — (Hi) of Definition 03 The next result gives a geometric characterization 
of viable sets. It will allow us to check that the sublevel sets of a control 
Lyapunov function are viable by means of the condition (iv) in Definition|3| The 
Nagumo-type geometric condition in the viability theorem is given in terms of 
the following second order normal cone to a closed set K C M N , first introduced 

in ma, 

:= {(p, Y)£R N x S(N) : for y -> x, y £ K, 

p ■ (y - x) + -{y - x) ■ Y(y -x)> o(\y - x\ 2 ) } 
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where S(N) is the set of symmetric NxN matrices. Note that, if (p, Y) £ N] l (x) 
and x £ OK, the vector p is a generalized (proximal or Bony) interior normal 
to the set K at x. In particular, if dK is a smooth surface in a neighborhood of 
x, p/|p| is the interior normal and Y is related to the second fundamental form 
of dK at x, see jlOj . 

Theorem 8 (Viability theorem [llj). Assume and Then a closed 
set K C ]R N is viable for (CSDE) if and only if 

Vx £ dK, V(p, y) £JVk(i), 3a e A : /(x, a) • p + trace [a(x, a)y] > 0. (12) 

The second tool for the proof of the Lyapunov-type Theorem is the following 
lemma on the change of unknown for second order partial differential equations. 
It says that the Hamilton-Jacobi-Bcllman inequality in condition (JJJ) in the 
definition of control Lyapunov function behaves as a geometric equation if the 
unknown satisfies also the condition in J7J of orthogonality between its gradient 
and the columns of the dispersion matrix a. We refer the interested reader to 
the chapters by Evans and by Souganidis in the book [H] for an introduction to 
the geometric PDEs of the theory of front propagation. 

Lemma 9. Let v satisfy condition for all (p, Y) £ J 2 '~V(x), x £ St N \ {0}. 
Let <j) be a twice continuously differentiable strictly increasing real map. Then 
w = 4> o v is a viscosity supersolution of 

max {-DV(x) ■ f(x, a) - trace [a(x, a)D 2 V(x)] } = (13) 

Proof. It is easy to check that, if (p, Y) £ J' 2, ~w(x), then 

(ip'(w(x))p,ip'(w(x))Y + ip"(w(x))p®p) e J 2 ^v(x), 

where ij) is the inverse of 4> and p (g> p is the NxN matrix whose entry is 
PiPj. Then, for (p,Y) £ J 2 '~w{x) and i^O there exists a such that 

{— ijj'{w{x))p ■ f(x,a) — trace [a(x, a) ■ (if)' (w(x))Y + ifj"(w(x))p <g> p)]} > 0. 

and 

ib"(w(x)) 

trace [a(x, a) ■ if)" (w(x))p ® p] = - v / " \a(x,a) T ^ (w(x))p\ 2 = 0. 

(tp'(w(x))) z 

Therefore 

—if>' (w(x))p ■ f(x,a) — trace [a(x,a) ■ ip (w(x))Y] > 
and we can conclude that 

sup {— p • /(x, a) — trace [a(x, a) ■ Y]} > 0. 

□ 

Proof, of Theorem\^ We begin with the proof of (ii). We fix an arbitrary fi > 
and consider the sublevel set of the function V 

K := {x\V(x) < fi}. 
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We claim that K is viable. Then for all initial points x G JR there exists 
a. E A x such that the associated trajectory X . satisfies 



which gives the estimate Ijlljl. Then the system is a.s. stabilizable and Lagrange 
stabilizable because 7-f 1 o 72 € /Coo- 

To prove that K is viable we will check the condition ljl2Jl of the Viability 
Theorem El For a given A > we define the nondecreasing continuous real 
function 



We claim that the function ip\ o V is a viscosity supersolution of equation 113|) 
for every A. To prove the claim we choose a sequence tp n of strictly increasing, 
smooth real maps that converge uniformly on compact sets to Then, for 
every n, the map ip n oV is a viscosity supersolution of equation (|13|) by LemmaEl 
By the stability of viscosity supersolutions with respect to uniform convergence 
we get the claim. 

Next we observe that the net ipx V is increasing and converges as A — > +00 
to the indicator function 



Viscosity supersolutions are stable with respect to the pointwise increasing con- 
vergence (see, e.g., Prop. V.2.16,p. 306, of |7|). Therefore the indicator function 
C of K is a viscosity supersolution of equation 113( 1 . From the definitions it is 
easy to check that 



By plugging this formula into the equation (|13|l we obtain exactly the condition 
((12(1 of the Viability Theorem and complete the proof of (ii) . 

To prove (i) we choose ~JL > small enough so that K :— {x G O : V(x) < 
/i}, for fi < /I, is closed in M N (for instance, Ji < inf ye ga liminf^^j, V(x)). 
Then the preceding part of this proof gives the viability of K and the estimate 
(lllfl for all x such that V{x) < ~p. Therefore, for some 5 > 0, holds for all 
x with |x| < 5 , and this gives the a.s. stabilizability of the origin. □ 

Next we give the proof of Theorem about asymptotic stability. It is ob- 
tained by first applying the previous Theorem [5] to a new system with an extra 
variable, and then using martingale inequalities as, e.g., in |17| . 

Proof, of Theorem^ We consider the differential system 



with initial data Xq = x and Zq = 0. We rewrite this system in JR N+1 as 



Ji(\X t \)<V(X t )<V{x)< 7l (\x\), Vi>0 a.s., 





J 2 --C{x) = -M%(x), VxedK. 



dX t 
dZ t 



f{X t ,a t )dt + a(X t ,a t )dB t 
l(X t )dt 




d(X t , Z t ) - f(X t , Z u a t )dt + a(X t ,Z t ,a t )d(B t ,0), t > 0, 
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where f(x, z, a) — (f(x, a), l(x)) and cr(x, z, a) — (cr(x, a), 0). Clearly it satisfies 
the conditions (J2J and (J5J. Let us consider the function 

W(x, z): OxR^ R 

(x,z)\ — ► V(x) + \z\ 

We claim that it is a Lyapunov function for (CSDE2). In fact, W is positive 
definite (because W > and W — only for (x,z) — (0,0)); W is lower 
semicontinuous, continuous at (0, 0) and proper since V is so. We have only to 
prove that W satisfies condition J2J. Fix x ^ and (x,z) with z > and a 
smooth function </> such that W — <f> has a local minimum at (x, z), i.e., 

V(a;) + z — 4>(x, z) < V(y) + w — <fi(y, w) 

for every (y, w), w > in a neighborhood of (x, 2). If we choose w = z we get a 
minimum in x for the function V(-) — </>(-, z), therefore (D x <f>{x, z),D%, x <j){x, z)) S 
J 2,_ V^(a;); if we choose y = x we find a minimum in z for the smooth function 
w 1 — ► it) — cj)(x,w), so D z (j)(x,z) — 1. Then there exists a € A such that 
(cr(x, a), 0) T (Da;</>(x, z), 1) = and 



{— D<j>[x, z) ■ f(x, z, a) — trace [a(x, z, a)D 2 (f)(x, z)l } = 

D 2 (t>(x,z) 



-[D ,<>(.,: :). I) ( / ^" ; J -/n/rr 



a (a;, a) 




= {— D x (f>(x, z) ■ f(x, a) — trace [a(x, a)D 2 x (j)(x, z)] } — i(x) > 0, 

since V is a strict Lyapunov function. Now fix (x, z) with z < and let be a 
smooth function such that 

V(x) - z - <j>{x, z) < V(y) -w- <j>(y, w) 

for every (y, w), w < in a neighborhood of (x, z). We argue as before and now 
get that there exists a € A such that (a(x,a), 0) T ■ (D x <p(x, z), —1) = and 

—D x (f>(x, z) ■ f(x,a) — trace [a(x,a)D 2 x ip(x, z)l + l{x) > 

> —D x <t>(x, z) ■ f(x,a) — trace \a(x,a)D xx <f>(x, z)l — l(x) > 0. 

because / is positive and V is a Lyapunov function. 

Finally, we consider (x,0) and a smooth function <fi such that 

V(x) - 4>(x, 0) < V(y) -w- <P(y, w) 

for every (y, w), w < in a neighborhood of (x, 0) and 

V(x) - <p(x, 0) < V(y) +w- cj)(y, w) 

for all (y, w), w > in a neighborhood of (x, 0). Then (D x <f){x, z),D xx (p(x, z)) g 
J 2 -~V(x), D z (/)(x,0) > -1, and D z <j>{x,0) < 1. Therefore there exists a € A 
such that (cr(x, a), 0) T • (D x <j>{x, z),D z <f)(x, z j) = and 

{— D(j){x, z) ■ f(x, Z,a) — trace [a(x, z,a)D 2 (f)(x, z)l } = 
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= {— D x (j){x, z) ■ f(x, a) — trace [a(x, a)D 2 x <j)(x, z)] } — D z <p(x, z)l(x) > 

= {—D x (f>(x, z) ■ f(x,a) — trace [a(x,a)D xx (j)(x, z)] } — l(x) > 0, 

This completes the proof of the claim, so we can apply Theorem GD to get 
for every x £ O an admissible control a. £ A x such that the corresponding 
trajectory (X . , Z .) of (CSDE2) with initial data (x, 0) remains almost surely in 
the level set K = {(y, w) € O X M\W(y,w) < W(x,0)}. Then, for all i > 
and almost surely, X t € O, 



W(X U Z t ) = V(X t ) + Z t = V(X t ) + / l(X s )ds < W{x, 0) = V(x), 

Jo 

and 

< V(X t ) < V(x) - f l(X s )ds. (14) 
Jo 

In particular, since I > 0, for some r > 0, \X t \ < r for all t almost surely. 

Next we claim that l(X t ) — » almost surely as t — > +oo. Let us assume 
by contradiction that the claim is not true: then there exist e > 0, a subset 
f2 e C fl with P(S! £ ) > 0, and for every u> G fl £ a sequence i„(w) — > +oo such 
that Z(X tri (w)) > e. We define 

F(r) := max \f(x,a)\, S(r) = max ||<r(x, a)||. 



We compute 



E<j sup \X S -X t \ 2 

t<s<t+h 



= e( sup | / f(X u ,a u )du+ [ a(X u ,a u )dB u \ 2 \ < 

lt<s<t+h Jt Jt J 

<2e( sup |/ /(X^a^cH 2 } +2e( sup \f <j(X u ,a u )dB u \ 2 \ < 

lt<s<t+h Jt ) lt<s<t+h Jt ) 

< 2F 2 (r) h 2 +2e( sup \[ a(X u ,a u )dB u \ 2 \ =: K. 

lt<s<t+h Jt ) 

By Theorem 3.4 in |18| (the process | a(X u ,a u )dB u \ is a positive semimartin- 
gale) we get 

K<2F 2 (r)h 2 +8 sup e(|/ <t(X u , a u )dB u A 

t<s<t+h L Jt J 

and by the Ito isometry 

K < 2F 2 {r)h 2 +8E jjf |ct(X u , S„)| 2 du j < 2F 2 (r)h 2 + 8£ 2 (r)/i. 

Then, Chebyshev inequality gives 

p f |T ^ , ^ ,\ . E{su Pt<s<t+/t |X s -X t | 2 } 
P<^ sup \X S -X t \>k\< — i - -< 

Lt< s <t+h J ^ 
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2F 2 (r)h 2 + 8Z 2 (r)h 
~ P ■ 

Since I is continuous, we can fix 6 such that \l(x) — l(y)\ < § if \x — y\ < 8 and 
| | , 1 2/ j < r. We define 

C := < ijj E £1 : sup \X a — x\ < 6 

L 0<s<h 

and choose < k < P(f2 e ) and /i > depending on <5 and e such that 
By the uniform continuity of / then the set 



B:= well : sup |Z(X S ) - Z(x)| < e/2 

I 0<s</i 

contains C and then 

p x (B)>i + k-p{n £ ). (15) 

From the inequality l)14[l. letting i — > oo, we get 



V(x)>B x l{X s )ds> / / Z(X s )dsdP> / V/ Z(X s )c2sdP> 

> [ V/i inf l(It)>/iV / inf Z(X t )dP > 

in, „ [tnH,t»(«)+fc] „in„ [t»H,t„H+fc] 



sup |Z(X 8 ) - ?(x)| < e/2 | a? = JC tn n « e 

0<s</i 



by the strong Markov property of the solutions of (CSDE) the estimate l|15|> 
gives P (sup < s < ft \l(X s ) - l(x)\ < e/2 | x = X tn ) > 1 + k - P X (Q £ ) for every 
n. Therefore P [(sup < s < h \l(X s ) - l(x)\ <e/2\x = X tn ) n fi e ] > k for every 
n: so, by the previous inequality, we get 

V(x) > /i^|fc = +oo 

n 

This gives a contradiction: then, for every e > 0, P(0 E ) = 0. We have proved 
that l(X~i) — + almost surely as t — > +oo, now the positive dcfinitcncss of I 
implies that \X t \ — ► almost surely as t — > +oo. □ 

Remark. If the function I is only nonnegative semidefinite the proof of the 
last theorem gives, for any x, a control a, whose trajectory X t satisfies a.s. 
V(X t ) < V(x) and l(X t ) as t -> +oo. Then the set C := {y \ l(y)_= 0} is an 
attractor, for a suitable choice of the control, in the sense that dist(X t , C) — > 
a.s. as < — * +oo. For uncontrolled diffusion processes results of this kind can 
be found in |35| and |17| and they are considered as stochastic versions of a 
theorem by La Salle. The earlier paper of Kushner 31 studies also a stochastic 
version of the La Salle invariance principle, namely, that the omega limit set of 
the process is an invariant subset of £, in a suitable sense. 
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4 A.s. feedback stabilization of affine systems 



In this section we give a result on the feedback stabilizability of systems afhne in 
the control in the case there exists a smooth strict control Lyapunov function. 
It is an analogue for the a.s. stability of a celebrated theorem of Artstein J2J 
and Sontag g0| for deterministic systems, extended by Florchinger [20] to the 
stability in probability of controlled diffusions. 

We begin with the simple case of a single-input affine system with uncon- 
trolled diffusion, that is, 

dX t = (f(X t ) + a t g(X t )) dt + a(X t )dB u (16) 

where f,g,cr are vector fields in M N with /(0) = and <t(0) = 0, B t is a 1- 
dimensional Brownian motion, and the control a t takes values in IR. We seek a 
function k : M N — > M, at least continuous in 2R \ {0}, such that the origin is 
a.s. asymptotically stable for the stochastic differential equation 

dX t = (f(X t ) + k(X t )g(X t )) dt + a(X t )dB t . (17) 

Then k is called an a.s. asymptotically stabilizing feedback for the control system 

If there are no constraints on the control, a smooth strict control Lyapunov 
function V satisfies, in M N \ {0}, 



f ■ DV + trace 



laa T D 2 V 



inf {ag ■ DV} < -I, a ■ DV = 0. 



Set j(x) := f ■ DV + trace [a<r T D 2 V~\ /2 + 1/2 and observe that the inequality 
for V means 

g(x)-DV(x)=0 => 7(1) < -l(x)/2 < 0. 

It is clear that k(x) := - f(x)/g(x) ■ DV(x), k{x) := if g(x) ■ DV{x) =0, could 
be a stabilizing feedback, but it is discontinuous where g{x) ■ DV{x) vanishes. If 
this case occurs we build a continuous feedback by means of Sontag's universal 
formula 001 1 i- e -i 



, 1 (x) + ^(x) + (g(x)-DV(x)r tf ,, nv ,,, n 
k(x):= g{x) DV(x) , i{g(x).DV(x)^0, (18) 

and k(x) = if g(x) ■ DV(x) = 0. By the argument in gD] k £ C{M N \ {0}) 
if V £ C 2 {1R N \ {0}), and k £ C^M 1 * \ {0}) if f,g,l are of class C 1 and 
V £ C 3 {1R N \ {0}). Moreover 



(/ + kg) ■ DV + trace 



-aa T D 2 V 
2 



< — , (i-DV = 
~ 2 



in Ml \ {0}, so V is a strict Lyapunov function for (|17(l and the origin is a.s. 
asymptotically stable. In conclusion, k is a stabilizing feedback for the affine 
control system ljTfi)l. 

If the control must satisfy a hard constraint, say a € [—1, 1], it is not hard 
to check that k(x) can be used in a neighborhood of the origin provided that 
DV and D 2 V are bounded near and either g(x) — > or DV(x) — > as x — > 0. 
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Next we use the same idea for the more general system with both the drift 
and the diffusion terms affine in the control 



dX t = ^f{X t ) + 49i{X t )\ dt + (a(X t ) + af r(X t )) dB t . 



(19) 



where /, <?i, cr, t are vector fields in JR N , B t is a standard 1-dimensional Brownian 
motion, and the controls a\, i = 1, P, are M- valued. The existence of a strict 
control Lyapunov function V implies that for some real number r the vector 
a + rr is orthogonal to DV, so r • DV ^ at all points where a ■ DV ^ 0, and 
we can define for all x € M N \ {0} 



h(x) := 



if a(x) ■ DV{x) = 0, 

r(l).fll'(l) 



. °W- DV W ifa{x)-DV(x)^0. 



Proposition 10. Assume the system $19\) has a strict control Lyapunov func- 
tion V S C 2 (M N \ {0}) and the function h is continuous in M N \ {0}. Then 
there exists continuous functions ki : M N \ {0} — > M, i = 1, ...,P — 1, such that 
(ki(x),...,kp-i(x),h(x)) is an almost surely asymptotically stabilizing feedback 
for the system ifHfl) . 

Moreover, ki(x) 6 [—1, 1] for x in a neighborhood of if DV and D 2 V are 
bounded near 0, and either DV(x) — > or gi(x) — * for all i as x — > 0. 

Proof. We recall from 0U| that the function <p(a, 0) := for a < 0, <j>(a, b) :— 
(a + \Ja 2 + b 2 )/b is real-analytic in the set S :— {(a, b) € 1R 2 : b > or a < 0}. 
We set 



70*0 := 
f{x) ■ DV(x)+ trace 



(a(x) + h(x)T(x))(a{x) + h(x)r(x)) 



T D 2 V{x) 



2 • 



p-i 



(3(x) :=Y.^)-DV{x)f 



i=l 

Since V is a strict control Lyapunov function, 

p-i 



71 



(x) + inf V otigdx) ■ DV(x) < 



l(x) 



so, for x 7^ 0, 

f3(x) =0 => j(x) < -l(x)/2 < 0. 
Therefore (7(2;), P(x)) £ S. Now we define, for i = 1, P — 1, 

fci(x) :=-^(7(aj),)8(x)) ffi (a:)-DV(a;), a; f 0, 

and fc(0) = 0. Then (k\{x), ...,kp^i(x),h(x)) is continuous in M N \ {0} and it 
satisfies 



( p-i \ 

/ + 

^ i=l / 



£>y + trace 



[a + hr)(a + hr) J 



D 2 V 



7 -^(7,/3) = -V / 7 2 +/? 2 <0. 



14 



Since (a + hr) ■ DV = by definition of h, V is a strict Lyapunov function for 
the equation 

dX t = (^f(X t ) + h(Xt)9i(X t )j dt + (a(X t ) + h{X t ) T (X t )) dB t . 

Therefore the origin is a.s. asymptotically stable for this equation. 

Finally we check the boundedness of k in a neighborhood of 0. This is trivial 
for 0(x) = 0. If (5{x) ^ 

1 VP 

Since either DV — ► or g± — ► for all i, (3(x) — > as x — > 0. We fix 5 > such 
that (3(x) < 5 implies j(x) < 0, and then choose a neighborhood of the origin 
where f3{x) < 6. In this set \k(x)\ < ^ (3{x) -> 0. □ 

Remark. The proof above gives an explicit formula for the stabilizing feedback 
in terms of the data and the Lyapunov function V only, which reduces to 118|) 
if t = and P = 2. From the formula one sees that the feedback is C 1 in 
M N \ {0} if h, /, g, a, r and I are such and V € C 3 (M N \ {0}). 

Note also that the continuity assumption on h is automatically satisfied if 
t ■ DV is either always nonnull or identically 0. 

Finally, it is straightforward to extend the Proposition to the case of M- 
dimensional noise with independent Brownian components Bj, ■■■ : Bf I and dif- 
fusion term of the form Yli=p 1 { a i + a t T i) dBl, with u^Ti vector fields and 
a I scalar controls. 



5 Some variants and extensions. 

In this section we collect several remarks on other applications of our methods. 
We begin with the almost sure exponential stabilizability. It means that there 
exists a positive rate A such that for every initial data x there exists an admissible 
control a. € A x whose corresponding trajectory X . satisfies 

V(X t ) < e- xt V{x) a.s. 

Proposition 11 (a.s. exponential stabilizability). Under the assumptions 
0) and the null state is almost surely exponentially stabilizable for ( CSDE) 
if sufficient that the system admits there exists a control Lyapunov function V 
satisfying conditions (i), (ii), (Hi) in the Definition^ and for some A > 
(iv)' for every (p, Y) € J 2 '~V(x) there exists a £ A such that 

o~(x,a) T p = and — p ■ f(x.a) — trace [a(x,a)Y] — XV (x) > 0. 

Proof. We consider the system 

f dX t = f(X u a t )dt + a(X u a t )dB t 
\ dY t = dt 

with initial data Xq — x and Yq — 0, and the Lyapunov function W(x, y) — 
e Xy V(x). By applying Theorem we obtain the existence of a control a. such 
that the corresponding trajectory almost surely satisfies V(X t ) < V(x)e~ xt , 
which is the desired inequality. □ 
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Next we extend the results of Section 2 to the stabilizability of a general 
closed set M C M N . We denote with d(x,M) the distance between a point 
x € M N and M. 

Definition 12 (a.s. stabilizability at M). The system (CSDE) is almost 
surely (open loop) stabilizable at M if there exists 7 G K, such that, for every x 
in a neighborhood of M , there is an admissible control function a. G A x whose 
trajectory X. verifies 

d(X t ,M) < j(d(x, M)) V* > almost surely. 

If in addition, 

lim d(X t ,M) = a.s. 

t — >+oo 

the system is almost surely (open loop) locally asymptotically stabilizable at 
M. 

If these properties hold for all x G M N the system is a. s. (open loop) 
globally asymptotically stabilizable at M. 

Remark. If M is a.s. stabilizable, then it is viable for (CSDE). In fact, the 
definition gives for x € M and e > an admissible control such that almost 
surely d(X t ,M) < e for all t > 0. Then for such control and any A > almost 
surely E x J + °° d(X t , M)e~ xt dt < f and so 

r+oc 

inf E,, / d{X t ,M)e- xt dt = 0. 

The convexity assumption Q and an existence theorem for optimal controls |26| 
imply that the inf is attained, and the minimizing control produces a trajectory 
staying in M for all t > 0. 

Definition 13 (control Lyapunov functions at M). Let O be an open 
neighborhood of the closed set M . A function V : O — > [0, +00) is a control 
Lyapunov function at M for (CSDE) if 

(i) V is lower semicontinuous; 

(ii) there exists 71 € /Coo such that V(x) < ji(d(x, M)) for all x G O; 
(Hi) there exists 72 € /Coo such that ^(d(x, M)) < V(x) for all x € O; 

(iv) for all x € 0\M and (p,Y) G J <~V(x) there exists a G A such that 
condition holds. 

The function V is a strict control Lyapunov function at M if it satisfies 
conditions (i)-(iii) and 

(iv)' for some Lipschitz continuous I : O — > 1R, l(x) > for all x M, and 
(p, Y) G J 2 '~V(x) there exists a £ A such that condition fllA) holds. 

Now we can state the analogues of the First and Second Lyapunov Theorems 
for the a.s. stabilizability at M. Their proofs are easily obtained from the 
arguments of the Theorems El and El by using d(x,M) instead of \x\ and noting 
that conditions (ii) and (Hi) in the Definition 1131 sav that the sublevel sets of 
the Lyapunov function form a basis of neighborhoods of M. 

Theorem 14. Assume ^J), 0), and the existence of a control Lyapunov func- 
tion V at M . Then 
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(i) the system (CSDE) is almost surely stabilizable at M ; 

(ii) if, in addition, the domain O of V is all M N , for all x ^ M there exists 
a. G A x such that the corresponding trajectory X . satisfies 

d(X t ,M)<^ 1 ^ 2 {dist{x,M))) Vf > a.s. 

with 71,72 G ICoo from Definition Hfli in particular, if M is bounded, the system 
is also a.s. Lagrange stabilizable. 

Theorem 15. Assume O), 0), and the existence of a strict control Lyapunov 
function V at M . Then 

(i) the system (CSDE) is a.s. locally asymptotically stabilizable at M; 

(ii) if, in addition, the domain O of V is all 1R , the system is a.s. globally 
asymptotically stabilizable at M . 

Remark (Stochastic target problems and absorbing sets). A stochastic 
target problem consists of steering the state of the system (CSDE) in finite time 
into a given closed set T (the target) by an appropriate choice of the control. 
One of the objects of interest is the set of initial positions from which this goal 
can be achieved almost surely in a given time t. Wc define these reachability 
sets for t > as 

TZ{t) = {x G M N | 3a G A* : X t eT a. s.} 

We consider a target T containing and invariant for the stochastic system and 
we assume there exists a global strict control Lyapunov function V as defined 
in Q such that 

inf l(x) = L > 0. 

R N \T 

We are going to show that each reachability set TZ(t) lies between two sublevel 
sets of the Lyapunov function V. The arguments in the proof of Theorem [SI 
show that for every initial point x ^ T there exists a control a. G A x such that 
the first entry time f x of the corresponding trajectory in the target is almost 
surely bounded by 

r x <(v{x)-\niV{y)\/L. (20) 

In particular, since the target T is invariant, it is reached almost surely in a 
finite time, and such time is also uniformly bounded, T is an absorbing set for 
the system according to the terminology in ,5 U . Next, from the assumptions 
and inequality 1)20(1 we get 

{x G M N | V(x) < Lt + wfV(y)} C K(t). 

Using Chebyshev inequality and estimates of the same kind as in the proof of 
Theorem we can find also for every t > a positive number k(t) depending 
continuously on t such that 

TZ(t) C {x G M N | V(x) < k(t)}. 

Let us mention that Soner and Touzi [37| developed recently a PDE approach 
to stochastic target problems, see also and the references therein for some 
interesting applications to geometric PDEs and front propagation problems. 
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6 Examples 



We begin with an example of an uncontrolled system that does not have a 
continuous Lyapunov function but has a l.s.c. Lyapunov function and therefore 
is a.s. stable. It shows that allowing V to be merely l.s.c. in Theorem 
really increases the range of the applications. Our example is a variant of a 
deterministic one by Krasovskii |2H| , namely, 




Y t (X? + Y 2 )* S m 2 (_^) 



see [H] for a discussion of this and other deterministic examples. 

Example 1. We transform the previous system in polar coordinates and per- 
turb it with a white noise tangential to the circles C n := {(x, y) : \{x, y)\ = 
and nondegenerate between two consecutive circles: 



dp t = 
dO t = 



Pt sin2 ( 



-1 + pf sin( 



' t )sin 2 (^)' 



pi 

> t ) COS' 



dt - 



dt 



)sin 2 (^) 



sin 2 (^ 



dB t 



where B t is a 1-dimensional Brownian motion and a satisfies the hypotheses 
for the existence and uniqueness of the solution of the stochastic differential 
equation. As in the undisturbed case, the circles C n are a.s. invariant and any 
point in C n is eventually reached a.s. by any trajectory starting in C n . Then 
any Lyapunov function V is constant on C n because V(pt,0t) < V(pq,9q) a.s., 
and c n := V\c„ ^ Cn-i '■— V\c n _ x at least on a subsequence. By property (iv) in 
the definition|3of Lyapunov function, for every (p, 9) in the interior of C n _i \ C n 

and every (p,X) S J 2 ~V(p,9), we get (a(p, 9) sin 2 (^), 0^ ■ p = 0. Since the 
diffusion is nondegenerate in the p direction in the interior of C n _i\C n , from the 
previous equality we deduce that, for such (p, 9), every element in J^ 2 '~V(p, 9) 
is of the form ((0,p2), X). This implies that the function V is constant in the p 
direction in the interior of C n _i \ C n and then it cannot be continuous. 

Now we check that the Lyapunov function of the undisturbed system in the 
unit ball does the job also for our perturbed stochastic system. We take 



V(p,9) :=-= 



for —= < p < —= 

yjn y/n - 



1 



V0. 



This is a positive definite function, lower semicontinuous and continuous at 0. 
We calculate its second order subjets and plug them into Q. If p ^ for all 

n, (p, X) S J 2: ~V(p,9) if and only if p = and X < 0, so the condition Q is 
trivially satisfied. 

On the other hand, (p, X) £ J 2 -^V(-k^ 1 9) if and only if 



P = 



s > and X = 



, c < 0. 



At the points with p = the drift / of the system is (0, —1) and the dispersion 
vector a is (0,0). Then 



f • p + —trace [er<7 T X] =0, a ■ p = 



IS 



and the condition JjJ is satisfied. Therefore Theorem [S] applies and the system 
is a.s. Lyapunov stable at the origin. 



The next two examples are about stochastic perturbations of stabilizable sys- 
tems. We consider a deterministic controlled system in M N 

X t = f(X t ,a t ) (21) 

globally asymptotically (open loop) stabilizable at the origin, i.e., asymptotically 
controllable in the terminology of deterministic systems ^2E2- By the converse 
Lyapunov theorem of Sontag |39l I42| . there exists a strict continuous control 
Lyapunov function for the system, i.e., for some positive definite continuous 
function L, a proper function V satisfying in M N \ {0} 

max{-f(x,a)-DV}-L(x)>0 (22) 

a£ A 

in the viscosity sense. (This is perhaps not explicitly stated in the literature; 
the original result of Sontag interprets this inequality in the sense of Dini 
derivatives of V along relaxed trajectories, the paper of Sontag and Sussmann 
[4"2~] in the sense of directional Dini subderivatives, and both these senses are 
known to be equivalent to the viscosity one, see, e.g., 03 13)- In the following 
examples we perturb in two different ways l|21|) and give condition under which 

V remains a control Lyapunov function for the a.s. stabilizability of the new 
stochastic system. 

Example 2. Consider the controlled diffusion process 

dX t = f{X u a)dt + a{X t )dB t (23) 

where B t is a Af-dimensional Brownian motion and a a Lipschitzean N x M 
matrix. Then 

V is a Lyapunov function for \2ty if, for some open set O 3 and some 
continuous I : O — > [0, +oo), V satisfies in viscosity sense in 0\ {0} 



-trace 



1 



-aa T D 2 V 



L — l>0, o~i ■ DV = Vi, (24) 



and it is a strict Lyapunov function if I is positive definite. 

In fact, this inequality and (|2"2")> give, for any (p, X) G J 2 '~V{x), 



max {—f(x, a) ■ p} — trace 

ottzA 



1 T V 

-co- X 



- I > 0, 



so V satisfies the inequality in condition (|l()(l , whereas the equality in condition 
(fTUf> reduces to <7j • p = 0. 

In the classical special case of V(x) — \x\ 2 and M = 1, the sufficient condition 
(|24ll for V to be a Lyapunov function of (|23|) reads 



l(x) := L(x) - \<r{x)y > 0, <t(x)-x = 0. 
For a noise of dimension M = N an example of a satisfying the orthogonality 

V(x)®DV 



condition in is a(x) = k ( I — DV , ) for any constant fc. 



19 



Example 3. Here we consider the perturbation of the deterministic system 
(I21|l by a function g of a if -dimensional diffusion process Y t : 

f X t =f(X u a t )+g(X t ,Y t ) 

\ dYt=b(Y t ,X t ,a t )dt + T(Y t ,X t ,at)dBt { ' 

where the function g : M n x M K — ► M n is Lipschitz continuous with g(0, y) = 
for all y, B t is a 1-dimensional Brownian motion, and b 1 r are vector fields in 
M K with the usual assumptions. We are still assuming that l|21[l has a strict 
control Lyapunov function V, i.e., 11' I'D holds with L positive definite. We are 
interested in the stabilizability of the perturbed system at the set M := { (x, y) £ 
IFL n x M K : x = 0}, which corresponds to the origin of the unperturbed system 
1)2 see Definition 1121 Note that the assumption on g implies the viability of 
M for ■ We claim that 

the function V, defined by V(x,y) := V(x) for all y, is a Lyapunov function at 
M for $25\) (see Definition \lS\) if, for some open set O90 and some continuous 
I : O x M K — * [0, +oo), V satisfies in viscosity sense in 0\ {0} 

inf {-g(x, y) ■ DV(x) - l(x, y)} + L(x) > 0, (26) 

y&lR K 

and V is a strict Lyapunov function if l(x, y) > for all x ^ and all y. 

In fact, since d((x,y), M) = |sg|, V satisfies the conditions (i) — (Hi) of the 
Definitional By f2^|l and lt2"H)l V is also a viscosity supersolution in O x M K \M 
of 

sup {-f{x, a) ■ DV(x)} - g(x, y) ■ DV(x) - l(x, y) > 0, 

a£A 

which is the inequality in (|10|l in this case, because V is constant in y. Finally, 
for the same reason, the condition in (|10|l of orthogonality of the diffusion vector 
to the level sets of V is trivially satisfied. 

The inequality (|26|l is a smallness condition of the component of g in the 
direction of DV with respect to L in the set 0, uniformly in y. For I = and 
V smooth in O \ {0} it becomes 

sup g(x,y)> DV(x) <L(x), tnO\{0}, (27) 
y em K 

which is satisfied, in particular, if 

sup \g(x,y)\ < L(x)/LipV, 

y<ElR K 

where LipV denotes the Lipschitz constant of V in O. We recall that, under our 
assumption that the deterministic system (1211) be asymptotically controllable, 
although V may not be smooth, it can be chosen semiconcave in R N \ {0} and 
therefore locally Lipschitz [SH]. If we make this choice, it is enough that the 
inequality (|27|l holds for all points x € O where V is differentiable, and the 
last inequality is guaranteed for all perturbations g with small sup-norm with 
respect to y. 

In the next two examples we give conditions on a radial function to be a 
Lyapunov function for a.s. stability. 
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Example 4. We consider as a candidate Lyapunov function for the general 
controlled system (CSDE) the function V(x) = v(\x\), for some smooth v : 
[0, +00) -> [0, +00) with v'(r) > for r > 0. Since W(x) = xt/(|x|)/|x|, in 
view of the orthogonality condition in 10 we restrict ourselves to controls a E A 
such that 

(Ti(x,a)-x = 0, Vi = l,...,M. (28) 

We compute 

r / s „■),,/ \ 1 w'(|x|) , . / „,. v'(|x|)\ |<t(x, a) T x| 2 

trace [a(x,a)D 2 F(x)l = , trace a(x, a) + U" x ^-r M ' 1 v ' ' 1 

Fl V Fl 

and use Ij28(l to obtain that V is a Lyapunov function if and only if, in a neigh- 
borhood O of 0, 

l(x) := max [—f(x, a) ■ x — trace a(x, a)] — ; — ; — > 0. 

aGA, cr(a;,a) T x=0 |X| 

i.e., 

min [f(x, a) ■ x + trace a(x, a)] < 0. (29) 

aGA, ct{x : ol) t x— 

This condition is independent of the choice of v. Moreover, if I > and Lipschitz 
in \ {0} and I — > as x — * 0, then V is a strict Lyapunov function. Note that, 
although the radial component of the diffusion must be null by J2HJ, its rota- 
tional component still plays a destabilizing role. In fact, tracea(x,a) > and 
whenever it is nonnull it must be compensated by a negative radial component 
of /. 

In particular, a single-input affine system with uncontrolled diffusion and 
1-dimensional noise B t 

dX t = (f(X t ) + a t g(X t )) dt + a(X t )dB t , a t G [-1, 1], 

has a radial Lyapunov function in O if and only if 

\cr(x)\ 2 

cr(x) • x = and \g(x) ■ x\ > f(x) ■ x + 1 y 1 in O, 
and V^(x) = |x| 2 /2 is a strict Lyapunov function in O if and only if 

l{x) := \g(x) ■ x\ - f(x) ■ x - > in O \ {0}. 

Moreover, k(x) := —sign(g(x) • x) is a stabilizing feedback if g{x) ■ x does not 
change sign; if it does k is discontinuous, then a continuous stabilizing feedback 
in a neighborhood of is given by the formula (|18(l in Section 4. 

Example 5. Here we study a system in M 2 written in polar coordinates (p, 8) 
and look for radial Lyapunov functions, i.e., of the form V(p, 6) = v(p). Consider 
the stochastic controlled system: 

(CSDE) { df>t = f(Pt^t,a t )dt + a(p t ,6 t , a t )dB t 
'\ d6 t = g(pt,0 t ,a)dt + T(p t ,6 t ,a t )dB t 

where all functions f.a,g,r are 27r-periodic and B t is (for simplicity) a 1- 
dimensional Brownian motion. The conditions for a function V = v(p) to be a 
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Lyapunov function of this system at the set M := {(0, 9) : 9 € M} are the fol- 
lowing. The orthogonality condition in i|10[l requires that for every (p, 9) there 
exists a subset A(p, 9) ^ of the control set A such that 

a(p,9,a)=0 VaeA(p,9). 

Then the condition is satisfied if v is a viscosity supersolution of the ordinary 
differential inequality 

sup {-f(p,9,a)-v'(p)}>0 

a£A(p,0) 

for p > and for each fixed 9 £ [0,2tt]. Of course the same result can be 
obtained from the previous example with some calculations based on the Ito 
chain rule. 



The last two examples are about the stabilization to sets M different from 
the origin, namely, the complement of a ball and a periodic orbit. 

Example 6. We consider the general system (CSDE) and the set 

M := {x | \x\ >R} = R N \ B R . 
We assume M is viable for the system. We take the radial function V 



V(x) 



R 2 -\x\ 2 \x\<R 
\x\> R 



and use the calculations of Example 01 to see that V is a Lyapunov function at 
M if and only if for every x with \x\ < R there exists a £ A such that 

ai(x,a) ■ x = Vi and f(x, a) ■ x + trace a(x, a) > 0. 

On the contrary of Example here the rotational component of the diffusion 
has a stabilizing effect. In fact, the drift f(x,a) is allowed also to point away 
from M if its negative radial component is compensated by the positive term 
trace a(x,a). 

If K C Br is a compact set and 

l(x) := max [f(x, a) ■ x + trace a(x, a)] > in Br \ 

a£A,ir(x 1 a) T x—0 

then M is locally asymptotically stable by Theorem^]and for all initial points 
x ^ K there is a control whose trajectories tend a.s. to M as t — + +oo. In this 
case we can say that K can be made almost surely repulsive by a suitable choice 
of the controls. In particular, we have a criterion of instability of an equilibrium 
point. 

Note also that if I > on dM = 8Br then for some control the trajectories 
starting in a suitable neighborhood of dM reach M in finite time a.s., as we 
observed in the last remark of Section 5. In particular, if I > in Br, then for 
every x € Br there exists a control a, such that the exit time of the correspond- 
ing trajectory X. from Br is almost surely bounded by (R 2 — \x\ 2 )/ min^ I. 
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Example 7. Consider (CSDE) in M 2 and assume the circle 7 := {x : \x\ = R} 
is a viable set. By the results of JI] this occurs if for all x <G 7 there exists 
a E A such that 

o~(x, a) ■ x = and f(x,a) ■ x + trace a(x, a) = 0. 

Then 7 is locally asymptotically stabilizable if, in a neighborhood {x : R — 
e < \x\ < R + e}, 

max [f(x, a) ■ x + trace a(x, a)] > if |x| < R, 

qGA, a{x ,a) T x—0 

min [f(x, a) ■ x + trace a(x, a)] < if \x\ > R. 

q£A, cr(x ,a) T x— 

This follows immediately from the arguments of the Examples 0] and |S| 
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